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Abstract 

Environments for decentralized on-line collaboration are now 
widespread on the Web, underpinning open-source efforts, 
knowledge creation sites including Wikipedia, and other ex¬ 
periments in joint production. When a distributed group 
works together in such a setting, the mechanisms they use for 
coordination can play an important role in the effectiveness 
of the group’s performance. 

Here we consider the trade-offs inherent in coordination in 
these on-line settings, balancing the benefits to collaboration 
with the cost in effort that could be spent in other ways. We 
consider two diverse domains that each contain a wide range 
of collaborations taking place simultaneously — Wikipedia 
and GitHub — allowing us to study how coordination varies 
across different projects. We analyze trade-offs in coordina¬ 
tion along two main dimensions, finding similar effects in 
both our domains of study: first we show that, in aggregate, 
high-status projects on these sites manage the coordination 
trade-off at a different level than typical projects; and sec¬ 
ond, we show that projects use a different balance of coor¬ 
dination when they are “crowded,” with relatively small size 
but many participants. We also develop a stylized theoretical 
model for the cost-benefit trade-off inherent in coordination 
and show that it qualitatively matches the trade-offs we ob¬ 
serve between crowdedness and coordination. 

Introduction 

In many settings on the Web, groups of people who may 
have no off-line associations with one another come together 
around a project-oriented site that supports remote interac¬ 
tion, discussion, and the production of a shared work prod¬ 
uct. This style of highly decentralized collaboration — the 
participants are geographically dispersed and may not inter¬ 
act with each other outside the context of the site — is the 
driving force behind a range of large open-source projects 
hosted on sites such as GitHub, as well as knowledge cre¬ 
ations sites including Wikipedia and recent experiments in 
massively collaborative problem-solving. 

One of the crucial questions that emerges, as these forms 
of interaction become increasingly influential, is to under¬ 
stand what makes them effective at a structural level, and to 
characterize the properties associated with better outcomes. 
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To address this question, we draw on a conceptual frame¬ 
work that has proven powerful in off-line domains — an¬ 
alyzing the effectiveness of teams through their coordina¬ 
tion mechanisms. These mechanisms broadly include the set 
of practices that help a team organize its collaboration on a 
task, for dividing up shared work, setting intermediate goals, 
and resolving disagreements (Barnard 1968; Cooper 1980; 
Foushee 1982; Helmreich et al. 1986; Malone and Crowston 
1990; Faraj and Sproull 2000). 

The Present Work: Trade-Offs in Coordination. A fun¬ 
damental property of coordination is the cost-benefit trade¬ 
off that it entails. Coordination is beneficial, but it comes 
at a cost — the work that team members put into coor¬ 
dination could be used on the substance of the project it¬ 
self (Becker and Murphy 1994; Entin and Serfaty 1999; 
Kittur, Lee, and Kraut 2009; MacMillan, Entin, and Serfaty 
2004). Referring to the communication overhead inherent in 
coordination, Macmillan et al. write, 

“Because communication is essential to team perfor¬ 
mance, effective team cognition has a communication 
overhead associated with the exchange of information 
among team members. Communication requires both 
time and cognitive resources, and, to the extent that 
communication can be made less necessary or more 
efficient, team performance can benefit as a result” 
(MacMillan, Entin, and Serfaty 2004). 

Our understanding of this cost-benefit trade-off comes 
largely from the study of relatively small face-to-face teams, 
as in the research noted above. But the trade-offs involved 
in coordination are equally or more pronounced in on-line 
domains due to the limited ability of on-line teams to rely 
on less costly implicit coordination mechanisms (Entin and 
Serfaty 1999; Kittur, Lee, and Kraut 2009; Salas, Stout, 
and Cannon-Bowers 1994; Wang et al. 1991), which of¬ 
ten require shared mental models that are difficult to main¬ 
tain on-line (Cannon-Bowers, Salas, and Converse 1990; 
Wittenbaum, Stasser, and Merry 1996). 

These trade-offs have clear implications for the design of 
these systems. There is thus an opportunity to combine the 
work in the organizational literature on coordination and its 
consequences for performance with the long line of work 
on coordination and its uses in on-line domains (Kittur and 



Kraut 2008; Kittur, Lee, and Kraut 2009; Kittur et al. 2007b; 
Krieger, Stark, and Klemmer 2009; Malone and Crowston 
1990) and to evaluate the findings in the context of some of 
the most active on-line collaborative settings. 

Research Questions for Coordination. An important 
question in the literature on off-line domains has been to 
understand the possible levels of coordination that balance 
the trade-off between cost overhead and performance gains, 
leading to the best team and individual performance. When 
we move to the context of large on-line projects, how do 
such projects manage this trade-off, and can we identify 
principles in how this balance is managed? 

We approach these questions both through the develop¬ 
ment of an analytical framework and through the study of 
large on-line datasets. In particular, we focus on Wikipedia 
articles and GitHub projects as two rich collaborative do¬ 
mains that share some essential abstract properties; 

• They each contain projects in which there is a primary 
work product and also a channel for coordination among 
members of the project team. 

• Participation in projects has an interlocking structure, in 
the sense that the participants in one project may also be 
involved in others. 

• Certain projects may have a higher level of status or visi¬ 
bility than others; on Wikipedia certain articles are fea¬ 
tured, and on GitHub certain projects can have a non¬ 
trivial number of watchers. 

These ingredients also relate in interesting ways to the co¬ 
ordination framework proposed by Malone and Crowston 
(Malone and Crowston 1990), who identify goals, activities, 
actors, and interdependencies as the general components of 
coordination. 

We organize our work around two central trade-offs in co¬ 
ordination; 

• Do high-status projects manage the coordination trade-off 
differently from typical projects? 

• How does coordination relate to a project’s team composi¬ 
tion and crowdedness — in particular, the amount of work 
produced relative to the number of members in the team? 

In addition to the data analysis supporting these two ques¬ 
tions, we develop a mathematical model to capture the bal¬ 
ance of costs and benefits in coordination abstractly. 

It is important to note a few features of our approach. 
First, we organize our work around the aggregate analysis 
of large datasets, and our findings are correspondingly ori¬ 
ented toward trade-offs at this cumulative level. Our inves¬ 
tigation is thus complementary to more fine-grained studies 
of individual projects and the specifics of their coordination 
strategies. Second, while our analysis is performed on two 
particular domains — Wikipedia and GitHub — we seek to 
articulate a framework that can be applied to on-line collab¬ 
oration across many contexts. With this in mind, we develop 
the core components of our approach — projects and shared 
work, coordination mechanisms, interaction across projects, 
and measures of status and visibility — at a general level. 


and illustrate how they can be applied across these two dif¬ 
ferent contexts to yield closely related findings. We hope 
through this alignment of common structure to suggest a set 
of principles that can be used more broadly. 

Finally, we believe that our analysis of coordination raises 
a number of possible suggestions for design, as we discuss 
further in what follows. Applications supporting collabora¬ 
tion increasingly seek to steer groups of users toward ef¬ 
fective interaction, and coordination mechanisms can be a 
powerful component of this process. But the trade-offs in¬ 
herent in coordination make clear that it can be a non-trivial 
problem to determine whether a system should be guiding a 
group toward more or less coordination in a given situation. 
By understanding how levels of coordination naturally vary 
with the visibility of a project and the crowdedness of the 
setting, we can establish principles by which coordination 
mechanisms can be tuned based on the underlying context. 

Summary of Results. We establish results on the trade¬ 
offs in coordination for our main questions described above. 
We first show that there are significant aggregate differ¬ 
ences in the way that coordination is used in the higher- 
status projects on both Wikipedia and GitHub, relative to the 
use of coordination in typical projects. This suggests that 
properties related to coordination can be relevant to ques¬ 
tions of performance and visibility in our on-line setting. For 
Wikipedia, where the set of participants in a single project is 
often larger and more diverse, we delve further into the ques¬ 
tion by looking at how much the effort and coordination is 
concentrated on a small set of the most active participants — 
this connects to an argument due to Kittur and Kraut, who 
posit that a balance of effort in on-line tasks in which a few 
participants do most of the work can correspond to a form 
of implicit coordination (Kittur and Kraut 2008). 

We then explore the relationship between coordination 
and the composition of the project. We find that additional 
coordination is most useful when there are many team mem¬ 
bers and the task is small, resulting in a crowded environ¬ 
ment; it is most wasteful when the number of team members 
is small and the size of the task is large. As with our results 
on high-status projects, our findings here too are consistent 
in Wikipedia and GitHub. 

We supplement this analysis with a formal model for 
studying the trade-off between the cost of explicit coordina¬ 
tion and the benefits it brings to a project. The predictions of 
the model also suggest that crowdedness is a key parameter 
in the coordination trade-off. 

Wikipedia Data 

In this section and the next, we describe our two datasets — 
Wikipedia and GitHub. We begin with Wikipedia, which is 
the larger and more complex of the two, and the one where 
we are in some cases able to compute correspondingly more 
complex functions of the data. 

For our purposes, each Wikipedia article constitutes a 
project that is produced by the set of users who edit it. 
Wikipedia captures many of the basic features that one sees 
in on-line collaboration more generally, and for our purposes 



it also exhibits three desirable properties. First, since each 
article constitutes a project with its own internal life-cycle, 
we can observe the history of many projects in a common 
environment, exploring sources of variation among them. 
Second, Wikipedia contains explicit markers of success and 
failure for projects, including recognition of certain highly 
successful articles. Finally, Wikipedia has well-developed 
mechanisms for explicit coordination, along with extensive 
records of coordination for each article (Viegas et al. 2007; 
Keegan, Gergle, and Contractor 2012). Our data contains 
approximately 3.4 million articles, each with a discussion 
page. This corresponds to the entire edit history of English 
Wikipedia up to April of 2007, developed as a resource by 
Crandall et al (Crandall et al. 2008). 

We now describe how the basic ingredients of our frame¬ 
work manifest themselves on Wikipedia: we first consider 
coordination mechanisms, and then measures for the status 
and visibility of articles. 

Coordination Mechanisms on Wikipedia. We look at 
two kinds of interactions to measure coordination in 
Wikipedia: (i) discussion edits, and (ii) comments left on ar¬ 
ticle edits. Each article on Wikipedia has a discussion page 
that is used to discuss issues related to the editing of the ar¬ 
ticle such as planning, resolving arguments, and enforcing 
conventions (Viegas et al. 2007). We use the number of edits 
to discussion pages as a measure of how much effort editors 
spend explicitly coordinating. There is significant variation 
across articles in the amount of discussion-page editing and 
in aggregate we will see that the variation points to overall 
differences across different types of articles. 

There is a second widely-used form of coordination: 
When a user edits a Wikipedia article, she has the option of 
including a comment where she can briefly explain the na¬ 
ture of the edit. Leaving comments is often helpful for other 
editors because the comments allow them to easily identify 
the kinds of edits other users have contributed. Comments 
are similar to discussion-page edits in that editors use them 
to communicate about the editing of the article, but in con¬ 
trast with discussion edits, comments are much terser and 
thus tend to explain the nature of an edit without long dis¬ 
cussions. In this sense, we can think of comments as lying 
somewhere on the spectrum between explicit and implicit 
coordination, and more implicit than discussion-page edits. 

In some of the analysis we will consider both of these 
coordination mechanisms, but at other points we will focus 
on discussion edits, since they are the mechanism that allows 
for explicit coordination, including the ability to engage in 
back-and-forth interaction between multiple participants. 

Use of Discussion Pages and Comments in Wikipedia. 

While edit comments and discussion pages are meant to fa¬ 
cilitate the collaboration among Wikipedia editors to write 
high-quality articles, people can use these tools for any pur¬ 
pose. Eor example, as is often the case on the Web, it is pos¬ 
sible that discussion pages and comment could be used for 
spam or other unintended purposes. If this were the case, 
measuring coordination by the number of discussion edits 


and comments would be misleading. To address this issue 
and to further understand how people use discussion pages 
and comments, we read a small random sample of comments 
and discussion sections from our data and manually catego¬ 
rize them according to their purpose. 

Using a set of 100 randomly selected comments, we first 
construct a set of categories of purposes. Then, we sample 
a new set of 100 randomly selected comments and organize 
them into the determined categories. We use the same pro¬ 
cedure to categorize a randomly selected section from 100 
discussion pages. Tables 1 and 2 show the categories and the 
number of examples in each category for comments and dis¬ 
cussion pages, respectively. We observe that the majority of 
the sampled comments and discussions are directly related 
to the writing of the article. Any comment or discussion that 
was not relevant to the editing of the article was placed in 
the ’’Other” category. Out of the 100 categorized comments 
and discussions, only 11 and 6 fell in the ’’Other” category. 

This analysis suggests that edit comments and discussion 
pages are indeed mostly used by editors to coordinate their 
collaboration on Wikipedia. Having established that these 
tools seem to be employed to facilitate coordination, we use 
the number of comments and discussion edits as a proxy for 
how much explicit coordination the editors perform. 


Category 

Num 

Mentions section 

52 

Reverted edit 

14 

Minor edit 

19 

Added content 

14 

Removed content 

7 

Correction 

2 

Mentions other users 

14 

Other 

11 


Table 1: Categories of Wikipedia edit comments and num¬ 
ber of examples in each category. Categories are based on a 
random sample of 100 comments. Comments can belong to 
more than one category. 


Featured Articles. We now discuss a natural status mea¬ 
sure for articles on Wikipedia. The Wikipedia community 
chooses an article to feature every day through a peer re¬ 
view process', and according to their guidelines, such arti¬ 
cles among the very best in terms of professional standards 
of writing, presentation, and sourcing^. 

We are interested in comparing coordination practices be¬ 
tween highly successful articles and average articles. While 
every measure of success includes particular idiosyncrasies 
and potential biases, we believe that using the featured- 
article designation as a success measure has a number of 
clear advantages. In particular, rather than defining an ad 
hoc success measure ourselves, the set of featured articles 

*See http://en.wikipedia.Org/wikiAVikipedia:Peer_review for a 
description of Wikipedia’s peer review process. 

^See http://en.wikipedia.Org/wiki/Wikipedia:Featured_article_criteria 
for a description of the attributes a featured article must have. 




Category 

Num 

Justify 

Text edit 

14 

Change metadata 

4 

Suggest Action 

Specific text edit 

9 

Add content 

13 

Remove content 

4 

Change metadata 

7 

References 

Provide 

4 

Request 

16 

Question 

On article’s topic 

8 

On Wikipedia conventions 

5 

Copyright issues 

8 

Dispute claim in article 

12 

General discussion about article’s direction 

8 

Other 

6 


Table 2: Categories of Wikipedia discussion page sections 
and number of examples in each category. Categories are 
based on a random sample of 100 discussion pages. Discus¬ 
sions can belong to more than one category. 

is a clear success measure that Wikipedia’s own community 
has defined. This has the advantage that our success measure 
is likely to be compatible with the standards and goals of 
Wikipedia editors, and it is something that produces incen¬ 
tives among editors. It is certainly true that many very good 
articles are never featured, but the existence of the designa¬ 
tion allows us to define a concrete and very high standard of 
success for an article; whether it has been featured or not. 

GitHub Data 

GitHub is a Git repository service used by millions of 
people to collaborate on open source software projects. 
Even though GitHub is smaller and more specialized than 
Wikipedia, it also exhibits the properties that make it a use¬ 
ful testbed for coordination in decentralized collaboration. 
We use data obtained from GitHub Archive^, which pro¬ 
vides a record of various aspects of all public repositories. 
From these data we are able to capture specific metrics of a 
project’s visibility, size, and amount of coordination among 
collaborators. Our data contains all public projects that were 
actively developed during a three month period starting in 
May of 2012, which consists of about 300,000 projects. As 
with Wikipedia, we discuss how to develop measures of co¬ 
ordination and status for GitHub projects. 

Coordination Mechanisms on GitHub. When a user 
commits changes to a repository, others users have the op¬ 
tion of making comments or asking questions by issuing a 
commit comment. This feature allows collaborators to dis¬ 
cuss contributions and provide feedback. We use commit 
comments to measure coordination on GitHub to understand 
how people use comments on GitHub, we manually cate¬ 
gorize a small sample of comments. We follow the same 
methodology we use to categorize Wikipedia comments. 

^ http: //w w w.githubarchive .org/ 


Table 3 shows the categories and the number of examples 
in each category. We observe that commit comments are 
largely used to discuss issues directly related to the project, 
and hence serve as a reasonable measure of coordination. 


Category 

Num 

Coding suggestion 

48 

Code explanation by author 

3 

Showing appreciation for other’s work 

15 

Reporting bug 

13 

Question about other’s code 

19 

General programming question 

4 

Expressing disapproval of other’s work 

7 

Other 

15 


Table 3: Categories of GitHub comments and number of ex¬ 
amples in each category. Categories are based on a random 
sample of 100 comments. Comments can belong to more 
than one category. 


GitHub Watchers. In GitHub, users have the option of 
watching repositories they are interested in. During the time 
period we are analyzing, watching a repository was a way 
bookmarking projects of interest"*. Since GitHub is mainly 
used to develop open source software and we only consider 
public projects, it is reasonable to assume that having many 
watchers signals high visibility. Hence, we use the number 
of watchers a project has as continuous measure of status. 

Having now articulated how the basic ingredients of our 
framework are reflected in both the Wikipedia and GitHub, 
we turn to our central questions in order. 

Coordination in High-Status Projects 

In this section we investigate how the amount of coordina¬ 
tion varies with the status of a project. For Wikipedia, we 
compare coordination in featured and non-featured articles, 
and in GitHub we measure coordination as a function of the 
project’s number of watchers. 

As noted in the introduction, we interpret our analysis via 
the trade-off between the costs and benefits of explicit co¬ 
ordination (MacMillan, Entin, and Serfaty 2004); While a 
highly coordinated team of collaborators has the advantage 
that it can split tasks, resolve disagreements, and set goals 
effectively, a team that works with little coordinating has the 
advantage that it spends all its efforts working on the task 
rather communicating with team members. 

The x-Core. It is known that in many on-line settings, a 
few users are responsible for much of the content of the site 
(Kittur and Kraut 2008; Kittur et al. 2007a). We call this 
small group of users the core of the project. More concretely, 
we define the x-core of a project to be the smallest set of 
users that account for an x fraction of all the work; these are 
the most active participants on the project. As we increase x, 

*See https;//developer.github.com/changes/2012-9-5-watcher- 
api/ 




the cc-core gets larger as more participants get included, and 
finally the 1-core is the set of all participants. We define the 
size of a project’s x-core as the number of users it contains. 

The x-core can be defined for any collaborative project, 
but we focus here on its application to Wikipedia, because 
the projects there are large enough to show substantial varia¬ 
tion as we range over possible values of x. By contrast, most 
of the GitHub projects we analyze are smaller and more fo¬ 
cused, and since the x-core analysis is correspondingly less 
informative, we do not apply it in GitHub. 

On Wikipedia, a natural question is whether featured ar¬ 
ticles tend to have a larger or smaller x-core size than non- 
featured articles. Measuring work by the number of edits, 
we compute the fraction of editors that belong to the x-core 
of each featured article, which captures the extent to which 
a small fraction of individuals are doing most of the work. 

Figure 1 shows the median fraction of editors in the ar¬ 
ticle’s x-core for different values of x. Throughout the pa¬ 
per, statistical significance of the difference in medians us¬ 
ing the Mann-Whitney U test (Kruskal 1957) is indicated 
by the color of the dots: black (p-val < .001), green (p-val 
< .01), yellow (p-val < .05). We observe that featured ar¬ 
ticles have significantly smaller core sizes for most values 
of X. Having few editors in the core may be beneficial for 
an article by making coordination among the core easier. In 
the same spirit as this result, Kittur and Kraut found that 
Wikipedia articles benefit from having many editors as long 
as a few editors are responsible for most of the edits (Kit¬ 
tur and Kraut 2008). They propose that organizing in such 
a way that a few editors are responsible for most edits is an 
implicit form of coordination. 



Figure 1: Median fraction of editors in x-core for featured 
articles (red) and non-featured articles (blue). 


Coordination in Featured Wikipedia Articles. We now 

study how Wikipedia editors make use of explicit coordina¬ 
tion mechanisms to interact with each other, and how this 
operates differently in featured and non-featured articles. In 
this analysis, we consider the x-core for each x; this lets 
us consider both the full article (when x = 1), as well as 
whether coordination mechanisms are differentially used by 
the most active editors (for smaller values of x). 

We would like to compare featured and non-featured ar¬ 
ticles, focusing on potential differences in coordination be¬ 
havior. We do this by constructing a comparison set — a 
set of featured articles and a set of non-featured articles over 
which we can compare aggregate properties. Given the a pri¬ 
ori differences between featured and non-featured articles. 


such a comparison set needs to be constructed carefully so 
that the contrasts we identify are not simply consequences 
of surface-level differences that we already understand. 

In particular, we control for three different features in set¬ 
ting up the comparison set. First is the volume of edits: fea¬ 
tured articles tend to have a lot of editing activity before they 
are featured, and once they are featured they attract even 
more attention than they would get under normal circum¬ 
stances. Hence, when we compare editing behavior in fea¬ 
tured and non-featured articles, we control for the number 
of edits. Second, we control for the stage of development 
of the article at the time of its last recorded edit. Since ar¬ 
ticles on Wikipedia are created every day, our data contains 
well established articles that have been edited thousands of 
times as well as newer ones that have only been edited tens 
of times. Finally, a third factor we control for is the stage of 
Wikipedia as a whole at the time when the edits to an article 
were done. Conventions among Wikipedia editors gradually 
change over time, and the behavior of editors can be system¬ 
atically affected by new conventions or features added on 
Wikipedia. Thus, our comparison set of featured and non- 
featured articles comprises roughly the same distribution of 
time points from the history of Wikipedia. 

In summary, we construct two sets of articles with roughly 
the same distribution of edits that were generated at roughly 
the same time period — one of featured articles and one of 
non-featured articles. In the Appendix we provide the full 
details on exactly how this comparison set is constructed. 

Let’s now look separately at how discussion pages and 
comments are used by featured and non-featured articles, 
considering the x-core for multiple values of x. Given the 
coordination trade-off we discussed above, it is unclear 
whether featured articles should display higher or lower use 
of these coordination mechanisms. 

Figures 2(a) and 2(c) show that for all values of x, the x- 
core produces more discussion edits and comments in fea¬ 
tured articles than non-featured articles. Recall that our set 
of non-featured articles is constructed to mirror the activity 
level of the featured articles as measured by article edits, so 
the comparison in Figures 2(a) and 2(c) is effectively saying 
that there is more coordination per edit in featured articles. 

A distinct but related question is to consider which of the 
editors on an article are accounting for the coordination ac¬ 
tivity. In particular, there are two natural hypotheses: that 
the most active editors are overrepresented in the discussion 
as they coordinate; or, alternately, that the less active editors 
are overrepresented in the discussion while their more active 
counterparts do the work of writing the article itself. 

To investigate this, we define d{x) (respectively c(x)) to 
be the fraction of discussion-page edits (respectively com¬ 
ments) created by editors in the x-core, and we plot the dif¬ 
ferences d{x) — X and c(x) — x as functions of x. Note that 
d(l) — l = c(l) — l = 0by definition, and in the event that 
every editor contributed to the discussion pages and com¬ 
ments in proportion to their article editing activity, we would 
have d(x) — x = c(x) — x = 0 for all x. 

In Figures 2(b) and 2(d) we show plots of d{x) — x and 
c(x) — X respectively, averaged separately over featured and 
non-featured articles. The fact that these functions are posi- 




(a) Median number of discus¬ 
sion edits by a;-core vs. x 



(b) Median value of ci(a;) — a; vs. 
X. d{x) is the faction of discus¬ 
sion edits contributed by x-core 
editors. 



(c) Median number of com¬ 
ments by a:-core vs. x 



(d) Median value of c{x) — x 
vs. X. c(x) is the faction of com¬ 
ments contributed by a:-core ed¬ 
itors. 


Figure 2: Differences in the use of comments and discussion pages among editors of featured (red) and non-featured (blue) 
articles. 


live over all at < 1 shows that the more active editors (those 
who belong to the at-core for small x) are in fact overrep¬ 
resented in these coordination mechanisms. Moreover, this 
overrepresentation is particularly pronounced for the fea¬ 
tured articles, again suggesting some of the distinctive ways 
in which featured articles use coordination. There is also 
an interesting contrast between discussion-page edits and 
comments: c(x) — x is higher for featured articles over all 
a: < 1, while d(x) —x is higher for featured articles only for 
X < 1/2. It would be interesting to further explore how this 
difference relates to the lighter-weight nature of comments 
relative to discussion-page edits. 

Coordination in Highly-Watched Githuh Projects. We 

now look at the relationship between coordination and sta¬ 
tus on GitHub, keeping our discussion more brief for this 
dataset. Since our measure of status in GitHub is continuous, 
rather than comparing two sets of projects, we look at how 
the number of comments per commit changes with number 
of watchers. Figure 3(a) shows that the number of comment 
per commit increases with number of watchers — this trend 
points in the same direction as our Wikipedia analysis, with 
higher-visibility projects using more coordination overall. 

It is natural to ask whether the number of watchers is serv¬ 
ing purely as a proxy for the number of commits, but as Fig¬ 
ure 3(b) illustrates, the number of comments per commit is 
roughly constant as a function of the number of commits. 
Thus the trend we are seeing is not due to the number of 
commits, and this argues for the relationship between visi¬ 
bility (number of watchers) and the level of coordination. 

We thus see that both GitHub projects and Wikipedia ar¬ 
ticles with higher status spend more effort coordinating. As 
noted earlier, this suggests the implication that projects with 
higher visibility may be usefully guided in the direction of 
greater levels of coordination relative to typical projects. 

Coordination in Crowded Environments 

Having established that projects with different status and 
visibility in Wikipedia and GitHub can exhibit signifi¬ 
cant aggregate differences in their use of coordination, we 
now explore the differences in coordination among projects 
with different team composition. Our basic intuition is that 




(a) Median number of com- (b) Median number of com¬ 
ments per commit vs. number ments per commit vs. number 
of watchers of commits. 

Figure 3: Comments as a function of the number of watchers 
(GitHub). 


projects that involve a larger number of users require more 
coordination than smaller teams. However, since our data 
sets include projects that output different amounts of work, 
the size of a team should be considered in relation to the 
amount of work it produces. For a project with a fixed 
amount of work produced, we expect that the amount of co¬ 
ordination will increase with team size. Furthermore, for a 
project with a fixed team size, we expect that the amount of 
coordination will decrease with amount of work produced. 
In summary, our hypothesis is that more crowded projects 
(large team size and low production) will require more coor¬ 
dination than less crowded ones. This is because as projects 
become crowded by users, but the amount of work available 
does not increase accordingly, users will lose the ability to 
work separately on independent tasks, and more communi¬ 
cation will be necessary to coordinate multiple users work¬ 
ing on the same task. We now test our hypothesis on our data 
sets. 


Crowdedness in Wikipedia. We would now like to com¬ 
pare our hypothesis with what we observe in Wikipedia. We 
first need to define measures for the project’s amount of 
work produced, number of users, and amount of coordina¬ 
tion. To define reasonable representations of these parame¬ 
ters we take each Wikipedia article a and consider the users 
Ua who have made at least one edit to the article and one edit 
to the discussion page. These are users who have demon- 










strated awareness of the existence of the article’s discussion 
page. Since articles in our data are at different stages, we 
consider a constant number of initial edits to measure the 
number of editors interested in the article and levels of co¬ 
ordination. We record the timestamp Ta when the 100th edit 
by users in Ua was made. We let Sa be the set of users in 
Ua who contributed at least one of the first 100 edits. The 
Sa parameter represents the size of the team as it existed at 
a fixed point in time. We let Da be the number of discussion 
edits made before time T^. The Da parameter represents the 
amount of coordination exhibited by the team. Here, we only 
consider discussion edits since they are the more explicit co¬ 
ordination mechanism. Finally, we let Na be the eventual 
size of the article in bytes, which represents the amount of 
work produced by the editors. 

Figure 4(a) depicts the amount of coordination log(£)Q) 
in an article of size Na and Ua editors. It is drawn as a heat 
map, with the color corresponding to the value of log(Da)- 
Furthermore, in figure 4(b), we split articles into four cate¬ 
gories depending on whether — relative to the median arti¬ 
cle — they have a lower or higher number of bytes {Na) and 
a lower or higher number of editors (Sa)- We compare the 
number of discussion edits {Da) and the number of discus¬ 
sion edits per editor (^) among the four categories. 

Figures 4(a) and 4(b) show the general trend we hypothe¬ 
sized. The amount of coordination, measured by the number 
of discussion edits and discussion edits per editor, increases 
with the number of editors and decreases with the size of the 
article. Articles that are crowded with many users and low 
production exhibit the most coordination. 


Crowdedness in GitHub. We now explore the role of 
crowdedness in the coordination of GitHub projects. We 
measure amount of work produced, number of users, and 
amount of coordination analogously to how we measured 
them for Wikipedia, and then perform an analogous investi¬ 
gation of the relationship. For each GitHub project p we let 
Up be the users who committed at least once and contributed 
at least one comment. We record the timestamp Tp when the 
100th commit by users in Up was made. We let Sp be the set 
of users in Up who contributed at least one of the first 100 
commits. We let Cp be the number of comments made be¬ 
fore Tp. Finally, we let Np be the eventual size of the project 
measured by the total number of commits in the full history 
of the project. We use Np as the amount of work produced, 
Sp as the number of users, and log(C'p) as the “amount of 
coordination.” 

Since the GitHub data set is much smaller than Wikipedia, 
in order to see the change in coordination with project size 
and number of users, we split the projects into 100 bins by 
splitting the number of users {Sp) and the size of the projects 
{Np) into 10 bins by percentile. We then measure the amount 
of coordination (log(C'p)) within each bin. Figure 5(a) shows 
a heat map of coordination log(Cp) as a function of num¬ 
ber of users {Sp) and the size of the projects {Np). To ob¬ 
serve a numerical representation of the trend, we also split 
the projects into four categories depending on whether they 
have low or high number of users and project size, relative 
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(a) Heat map of “amount of 
coordination” (log(Da)) as a 
function of the number of “ar¬ 
ticle parts” {Na) and number 
of editors (Sa) in Wikipedia 
articles. 
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(b) Articles are split into 4 
categories at the median size 
(Na) and number of editors 
(Sa). Each area shows the 
median number of discussion 
edits (Da), median number 
of discussion edits per edi¬ 
tor {Da/So), and number of 
articles (N). Differences be¬ 
tween cells are statistically 
different (p-val < 0.05) 
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Figure 4; Amount of coordination in Wikipedia articles of 
different composition. 


to the median. Figure 5(b) shows the median values of Dp, 
and the number of projects in each category. 

We find that the trend is similar to the one observed in 
Wikipedia. As projects become more crowded with many 
users and small size, users tend to coordinate more. Overall, 
the fact that coordination and crowdedness align closely in 
both domains raises a further potential implication for de¬ 
sign — to recognize projects that are becoming increasingly 
crowded, and to correspondingly guide groups toward coor¬ 
dination resources as this is occurring. 

Modeling Coordination and Crowdednes 

We now develop a simple theoretical model of coordina¬ 
tion, so that we can study the cost-benefit trade-off in co¬ 
ordination at a more formal level. The model is highly styl¬ 
ized to reduce this trade-off to its basic essence. The benefit 
of working with a concrete stylized model is that by strip¬ 
ping away much of the complexity of the coordination pro¬ 
cess that is specific to Wikipedia and GitHub, we can high¬ 
light how certain trade-offs depend only on a set of sim¬ 
ple assumptions that are present in other domains outside 
of Wikipedia and GitHub. We find that the model accom¬ 
plishes this - from the analysis of the model, we identify the 
relationship between crowdedness and coordination, which 
matches our original hypothesis and the effects we found in 
Wikipedia and GitHub. 

General Setting for the Model. We begin by modeling 
the costs and benefits of coordination in a stylized setting 
that represents a team’s collaboration. The costs result from 
the fact that time spent on coordination does not directly ad¬ 
vance the project itself. The benefits will be based on divid¬ 
ing up the work without redundancy — in the absence of 
coordination, there is the danger that two people will try to 
do the same step in the project simultaneously, resulting in 
a loss in efficiency. In the context of GitHub and Wikipedia, 
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(a) Heat map of “amount 
of coordination” (log(Cp)) 
as a function of percentile 
of number of “project parts” 
(Np) and number of users 
(Sp) in GitHub projects 


IIIHI 
-| 121)0 
I 101)0 
81)0 
£; GDI) 

I 

I ™ 

"() 12 3 5 0 7 8 

Number of contributors of initial 100 commits 

(b) Split of projects into 4 
categories by low or high 
size (Np) and number of ed¬ 
itors (Sp). Each area shows 
the median number of com¬ 
ments (Dp), median num¬ 
ber of comments per user 
(Dp/Sp), and number of 
projects (N). Differences be¬ 
tween cells are statistically 
different (p-val < 0.05) 
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Figure 5; Amount of coordination in GitHub articles of dif¬ 
ferent composition. 
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(a) Values obtained from simu- (b) Values obtained from ana- 
lations. Parameter a = 1. lytical approximation. Parame¬ 

ter 0 = 1 . 



(c) Values obtained from ana¬ 
lytical approximation. Parame¬ 
ter a = 0. 


we can think of this model as capturing the way in which 
coordination is particularly important when two users are 
working on the same section of an article or the same part 
of a program. Without coordination or discussion, users who 
disagree on the outcome often engage in “edit wars,” undo¬ 
ing each other’s work (Viegas, Wattenberg, and Dave 2004). 

As discussed above, our model is not designed to cap¬ 
ture all the nuances encountered in contexts as complex as 
Wikipedia and GitHub. Indeed, our goal is in a sense the op¬ 
posite, to find the simplest formulation of a model in which 
the inherent trade-off between coordination and efficiency 
emerges from the basic properties of the model. 

The structure of our model is as follows. There is a project 
with N parts that need to be finished. Each part starts in the 
unfinished state, and requires one unit of work; a part tran¬ 
sitions from the unfinished to the finished state when a user 
works on it. A set of E users work on the project. In a sin¬ 
gle step, a user contributes a unit of work to one part. If the 
part is currently unfinished, then it becomes finished. If the 
part the user worked on was already in the finished state, the 
situation is more subtle, reflecting the collision between two 
users. The new contribution has no effect with probability 
1 — a; and with probability a it in fact has a negative ef¬ 
fect, clashing with the previous contribution and returning it 
to the unfinished state. This captures the idea noted above, 
that when two people work on the same task, it can create 
additional work as the differences are resolved. 

Users arrive sequentially and can either coordinate with 
previous users before contributing, or contribute without co¬ 
ordinating. Specifically, each user is allowed to perform two 
actions, selected in one of the following two ways; 

• Coordinate: If the user coordinates, she uses her first ac¬ 
tion to coordinate with others to find an empty project 
part, and then uses her second action to add a contribu¬ 
tion to the empty part, moving it to the finished state. In 
this case, she contributes exactly one part to the project. 


Figure 6: Predicted amount of coordination by the model. 
Heat maps show the amount of coordination (/3) that opti¬ 
mizes the number of finished parts of a project with E work¬ 
ers and N parts. 


• Not Coordinate: If the user does not coordinate, then 
she uses both of her actions to sequentially contribute to 
two randomly selected project parts. Either of these parts 
might turn out to be finished, in which case the contribu¬ 
tion has no effect (with probability 1 — a) or returns the 
part in question to the unfinished state with probability a. 

Each user chooses to coordinate independently with prob¬ 
ability /3 and to not coordinate with probability 1 — /3, where 
P is fixed in the beginning of the process. The two actions 
allowed to users in the model represent the ability that users 
in decentralized collaboration platforms have to spend all 
the effort contributing to the projects, or to split their efforts 
between contributing and coordinating. For example, as we 
observed, some editors justify their edits in the discussion 
page and some propose their edits before applying them to 
the article. Likewise, some GitHub users explain their code 
after committing it. 

Collectively, the set of users would like the project to have 
as many finished parts as possible. How much should they 
coordinate in order to maximize this objective? We are in¬ 
terested in finding the value of that maximizes the number 
of finished parts by the end of the process. 

Figure 6(a) depicts the optimal with N parts, E users, 
and with a — 1. It is drawn as a heat map, with the color 
corresponding to the value of fi. Here has been optimized 
by running a large number of simulations of the process. We 
observe first of all from the heat map that the relationship be¬ 
tween crowdedness and coordination in this synthetic model 
follows the same qualitative direction that we observed for 
both Wikipedia and Github. We now turn to an analytical ap- 







































proximation of the optimal /? and show it agrees very closely 
with the simulation results. 

Analysis of optimal coordination probability Fix N, E, 

and a, and let Pi be the expected number of finished parts 
the project has after i users have passed through the project. 
Also, let Xc,k be the probability that a user who does not 
coordinate finishes k parts given that there are C finished 
parts at the time the user arrives. When fc < 0, Xc,k is the 
probability that the user will undo k parts from the project. 
We make the following approximation; 

P,+i = (l-^)(F,+2Ap.,2+Ap.,i-Ap.,_i-2Ap.._2)+/3 

( 1 ) 

Note that this is an approximation since the exact value of 
P,+i is (1-/3)(Q+2 Ac,.2+Ac..i-Ac..-i-2Ac,.-2) + 
j3, where Ci is the actual number of finished parts after user 
i, not the expected value of Cp That is, we make the deter¬ 
ministic approximation that Ci is always its expected value. 

Writing the values of Xc,k in terms of a, C, and N and 
plugging them into equation 1, we get the following recur¬ 
rence relation: 

Pi+l = -A-Pi + Pq 

whereA= ^^(l + a)2-2^-^(l + a) + l (2) 

and Pq = - + a) + 2 — /3 

Solving the recurrence, we get a closed form solution; 

■ 4 * _ 1 ■ 

P^ = Po - ifA^l 

_A-l\ ^ (3) 

and Pi = iPo if A = 1 

For fixed N, a, and E, Pe in equation 3 gives the ex¬ 
pected number of completed parts as a univariate function 
of j3. We can easily optimize this function and solve for the 
value of /? that leads to the most completed parts. Figure 
6(b) shows the optimal value of (3 for a range of values of N 
and E and a = 1. We observe that the result from the simu¬ 
lations (figure 6(a)) and the analytical approximation (6(b)) 
are very similar, suggesting that both approaches lead to a 
good approximation of the optimal solution. 

We notice first that when E > N the best /? is 1, for 
the simple reason that in this case, if all users coordinate, 
they will finish all the parts - the best possible outcome. 
Conversely, when N is very large and E is small then the 
best P is close to zero. That is because there are many more 
parts than users, so it is unlikely for users to collide, and 
hence coordinating is mainly wasteful. In between these two 
extremes, there are values of N and E in which the best 
value for /3 lies non-trivially away from both zero and one. 

In the previous analysis we set a = 1. We now show that 
the high level predictions of the model hold for any value of 
a. Figure 6(c) shows the optimal /3 for a range of value of 
N and E using a = 0. We observe that the area where the 
optimal value of /3 is strictly between 0 and 1 rotates clock¬ 
wise and there is overall less coordination. However, the ba¬ 
sic trend observed when a = 1 holds here too — projects 


require more coordination when the number of project parts 
is smaller relative to the number of users. Analogous plots 
across various values of a between 0 and 1 exhibit the same 
general trends. For all a the model thus makes a basic quali¬ 
tative prediction: users should coordinate more as the project 
becomes more “crowded,” with E large relative to N. The 
alignment of this stylized model with our hypothesis and 
the trends we observed in Wikipedia and GitHub suggests 
the potential robustness of the relationship between crowd¬ 
edness and coordination in other collaborative domains. It 
is striking that this relationship emerges clearly from the 
model, despite the fact that crowdedness was not explicitly 
built into the model’s structure. 

Discussion 

Through the use of rich datasets and a theoretical model 
of coordination, we have analyzed the performance of on¬ 
line projects from the perspective of coordination mecha¬ 
nisms. On both Wikipedia and GitHub, we find that projects 
with high status and visibility differ in aggregate from other 
projects in the way that they use coordination. We also find 
that crowding of project participants is a key parameter un¬ 
derlying the coordination level in Wikipedia and GitHub. We 
develop a theoretical model for the coordination process; the 
analysis of the model aligns with the trends found in the 
data, suggesting the potential robustness of the findings. 

The relationship between coordination and these struc¬ 
tural properties of projects can suggest principles for design¬ 
ing coordination mechanisms in several dimensions. 

• The relationship between crowdedness and coordination 
suggests that coordination mechanisms should not be sur¬ 
faced uniformly across different projects, but instead em¬ 
phasized more strongly on crowded projects - those with 
many team members relative to the project’s size. 

• In a related vein, our analysis has pointed to differences 
between lightweight and heavyweight coordination mech¬ 
anisms, especially in their differential usage across active 
and peripheral team members. There is thus a need to in¬ 
tegrate these different coordination styles across different 
types of contributors. 

• Featuring and visibility interact in subtle ways with co¬ 
ordination, as we have seen; there may also be additional 
dimensions along which coordination should most effec¬ 
tively be varied. 

Broadly speaking, our framework here suggests that an un¬ 
derstanding of the roles of coordination mechanisms in dif¬ 
ferent settings can benefit from a data-oriented analysis of 
their inherent trade-offs. 

The similarities in these effects across both Wikipedia and 
GitHub suggests some of their generality; it will be interest¬ 
ing to consider how these findings carry over to other on¬ 
line domains in which decentralized teams collaborate on 
projects. As we have seen from the two domains in this pa¬ 
per, the basic ingredients of our model and analysis can usu¬ 
ally be directly adapted to new settings, since the framework 
can be applied whenever there is a group faced with a pri¬ 
mary work product and a separate channel for exchanging 



coordination-related messages. Ultimately, seeing how these 
findings transfer across domains will be a next useful step on 
the way toward understanding the process of large-scale on¬ 
line collaboration. 
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Appendix: Sampling Wikipedia Articles 

In this appendix we describe the sampling of featured and 
non-featured Wikipedia articles in more detail. Recall that 
our goal here is to create, for each featured article, a compar¬ 
ison set of non-featured articles that have roughly the same 
number of edits at roughly the same periods of time. 

Our procedure is as follows. For each article a, we let 
eb{a,y) be the number of times a was edited before year 
y. Similarly, we let ed{a,y) and ea{a,y) be the number of 
times a was edited during and after year y. For each arti¬ 
cle af featured in year y, we find k random non-featured 
articles, La.fc, that have approximately the same number of 
edits as a f during the years before and after y. That is, 
for an article a/ featured during year y, the non-feature 

article a„ can be in La^k if ^ qq 

\eaiaf,v)-ec,(a„,v)\ gg 
ea{af,v) 

We aim to investigate the differences in the amount of co¬ 
ordination between featured and non-featured articles that 
are not a direct consequence of differences such as volumes 
of edits. Hence, we also require that ei,{af,y) < et,(a„,t/) 
for o„ to be included in La.fc- The results turn out to be the 
same with and without this additional restriction. We choose 
the article a„ from the set of all non-featured articles without 
replacement. That is, the sets La.fc are pairwise disjoint. 

We define F to be the set of articles that have been fea¬ 
tured on Wikipedia for which Ta.fc in non-empty, and we 
define NF = Ua 6 F{Ta, 3 o}. the non-featured articles with 
approximately the same number of edits as a featured arti¬ 
cle during the years before and after the year the article was 
featured. Throughout the paper we compare the sets F and 
NF. We repeat the analysis with different choices of k and 
find that the results are consistent for all moderate values of 
k. We present the results for k = 30 
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